fix(bedrock_mantle): stop dropping the web_search tool on /v1/responses - #35987

TLDR

Problem this solves:

• Bedrock Web Search never ran through /v1/responses
• bedrock_mantle silently dropped the web_search tool
• Callers got ungrounded answers with no citations

How it solves it:

• Allow web_search through the Mantle tool filter
• Mark the GPT-5.4/5.5/5.6 Mantle entries as web search capable
• Regression tests pin the tool surviving both entry paths

User Flow

Before: a developer who sends Bedrock's web_search tool through the gateway gets a 200 with an ungrounded answer and no citations

1. They send POST https://litellm-domain/v1/responses with "model": "bedrock-gpt-5-6-sol", a question about a recent event, and "tools": [{"type": "web_search", "external_web_access": false}]
2. The 200 response echoes "tools": [], and its output holds only reasoning and message items, no web_search_call
3. The message text says the date "has not occurred yet" and carries no url_citation annotations
4. The same happens on POST https://litellm-domain/v1/chat/completions with "web_search_options": {} (no annotations on the message) and on POST https://litellm-domain/v1/messages with the web_search_20250305 tool (an ungrounded text block)

After: the same request runs the searches on Bedrock and comes back grounded with citations

1. They send the same POST https://litellm-domain/v1/responses with the web_search tool
2. The 200 response echoes "tools": [{"type": "web_search", ...}], and output holds two web_search_call items (a search and an open_page) ahead of the message
3. The message text summarizes the real announcement and carries two url_citation annotations pointing at aws.amazon.com
4. POST https://litellm-domain/v1/chat/completions with "web_search_options": {} now returns url_citation entries under annotations, and POST https://litellm-domain/v1/messages with the web_search_20250305 tool returns a grounded text block with the source link inline

Relevant issues

Fixes #36649

Linear ticket

Resolves LIT-5224

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/llms/bedrock_mantle/test_bedrock_mantle_responses_transformation.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Two live proxies against real Bedrock Mantle in us-east-1, each booted with --num_workers 2 on a random free port from its own worktree: Before at the merge base on port 22948, After at this PR's tip on port 27930. Same config, same payloads, same question, only the checked out commit differs. AWS_BEARER_TOKEN_BEDROCK comes from the environment

model_list:
  - model_name: bedrock-gpt-5-6-sol
    litellm_params:
      model: bedrock_mantle/openai.gpt-5.6-sol
      aws_region_name: us-east-1
  - model_name: bedrock-gpt-oss-120b
    litellm_params:
      model: bedrock_mantle/openai.gpt-oss-120b
      aws_region_name: us-east-1

general_settings:
  master_key: sk-1234

python litellm/proxy/proxy_cli.py --config mantle.yaml --port $PORT --num_workers 2

post() { curl -sS --max-time 300 -D "$2.headers" -o "$2.out" -w "HTTP %{http_code}\n" -X POST "http://localhost:$PORT$1" \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d @"$2.json"; }

Payloads, all asking What did AWS announce about Web Search on Amazon Bedrock on August 4 2026?:

// responses_nonstream.json
{"model":"bedrock-gpt-5-6-sol","input":"<question>","tools":[{"type":"web_search","external_web_access":false}]}
// responses_stream.json
{"model":"bedrock-gpt-5-6-sol","input":"<question>","tools":[{"type":"web_search","external_web_access":false}],"stream":true}
// chat_web_search_options.json
{"model":"bedrock-gpt-5-6-sol","messages":[{"role":"user","content":"<question>"}],"web_search_options":{}}
// messages_web_search.json
{"model":"bedrock-gpt-5-6-sol","max_tokens":2048,"messages":[{"role":"user","content":"<question>"}],"tools":[{"type":"web_search_20250305","name":"web_search"}]}
// responses_gptoss.json (a Mantle model AWS does not offer Web Search on)
{"model":"bedrock-gpt-oss-120b","input":"<question>","tools":[{"type":"web_search","external_web_access":false}]}

Before (4990f06)

/v1/responses, non-streaming

1. post /v1/responses responses_nonstream; jq -c '{types: [.output[].type], tools, citations: [.output[] | select(.type=="message") | .content[].annotations[]? | .url]}' responses_nonstream.out; grep -i '^x-litellm-response-cost:' responses_nonstream.headers
2. Observed: HTTP 200, the tool is gone and nothing was searched

{"types":["reasoning","reasoning","message"],"tools":[],"citations":[]}
x-litellm-response-cost: 0.0357445
answer: August 4, 2026 has not occurred yet, so AWS has not made an announcement on that date. The year may be a typo

/v1/responses, streaming

1. post /v1/responses responses_stream; grep '^data: ' responses_stream.out | sed 's/^data: //' | jq -r .type | sort | uniq -c | sort -rn
2. Observed: HTTP 200, no web_search_call events at all

  54 response.output_text.delta
   2 response.output_item.done
   2 response.output_item.added        (item types: reasoning, message)
   1 response.output_text.done
   1 response.in_progress
   1 response.created
   1 response.content_part.done
   1 response.content_part.added
   1 response.completed                (citations in the final response: 0)
final answer: August 4, 2026 has not occurred yet, so AWS has not made a verifiable announcement on that date.

/v1/chat/completions with web_search_options

1. post /v1/chat/completions chat_web_search_options; jq -c '{finish_reason: .choices[0].finish_reason, annotations: .choices[0].message.annotations, content: .choices[0].message.content[0:120]}' chat_web_search_options.out; grep -i '^x-litellm-response-cost:' chat_web_search_options.headers
2. Observed: HTTP 200, no annotations, ungrounded answer

{"finish_reason":"stop","annotations":null,"content":"August 4, 2026 has not occurred yet, so AWS has not made an announcement on that date.\n\nIf you meant **August 4, 2025**"}
x-litellm-response-cost: 0.0365365

/v1/messages with the web_search tool

1. post /v1/messages messages_web_search; jq -c '{types: [.content[].type], stop_reason, text: .content[0].text[0:120]}' messages_web_search.out; grep -i '^x-litellm-response-cost:' messages_web_search.headers
2. Observed: HTTP 200, ungrounded text, no source link

{"types":["text"],"stop_reason":"end_turn","text":"August 4, 2026 has not occurred, so there is no verifiable AWS announcement from that date yet. If you meant August 4, 2025"}
x-litellm-response-cost: 0.0189475

/v1/responses on a Mantle model without Web Search (gpt-oss-120b)

1. post /v1/responses responses_gptoss; jq -c '{types: [.output[].type], tools, answer: ([.output[] | select(.type=="message") | .content[].text] | join(" ") | .[0:120])}' responses_gptoss.out
2. Observed: HTTP 200, the tool is dropped and the model makes the announcement up

{"types":["reasoning","message"],"tools":[],"answer":"**AWS announcement, Web Search for Amazon Bedrock (August 4 2026)**  On **August 4 2026**, Amazon Web Services released a major update"}

After (8713ba8)

/v1/responses, non-streaming

1. post /v1/responses responses_nonstream; jq -c '{types: [.output[].type], tools, citations: [.output[] | select(.type=="message") | .content[].annotations[]? | .url]}' responses_nonstream.out; grep -i '^x-litellm-response-cost:' responses_nonstream.headers
2. Observed: HTTP 200, the tool is echoed back, Bedrock ran a search and opened the page, and the answer cites both

{"types":["reasoning","web_search_call","reasoning","web_search_call","message"],"tools":[{"type":"web_search","filters":null,"search_context_size":null,"user_location":null}],"citations":["https://aws.amazon.com/about-aws/whats-new/2026/08/amazon-bedrock-web/","https://aws.amazon.com/ko/blogs/machine-learning/introducing-web-search-on-amazon-bedrock-for-foundation-model-grounding/"]}
x-litellm-response-cost: 0.062165125
web_search_call: {"status":"completed","action":{"type":"search","query":"site:aws.amazon.com/about-aws/whats-new/2026 \"Web Search\" \"Amazon Bedrock\" \"August 4, 2026\""}}
web_search_call: {"status":"completed","action":{"type":"open_page","url":"https://aws.amazon.com/about-aws/whats-new/2026/08/amazon-bedrock-web/"}}
answer: On **August 4, 2026**, AWS announced the **general availability of Web Search on Amazon Bedrock** for OpenAI GPT models. Key points: Native, server-side web grounding ... Supported models: GPT-5.4, GPT-5.5, and GPT-5.6 Luna, Terra, and Sol.

/v1/responses, streaming

1. post /v1/responses responses_stream; grep '^data: ' responses_stream.out | sed 's/^data: //' | jq -r .type | sort | uniq -c | sort -rn
2. Observed: HTTP 200, three searches stream through as response.web_search_call.* events and the citations arrive as annotation.added events

 206 response.output_text.delta
   7 response.output_item.done
   7 response.output_item.added        (item types: reasoning, web_search_call, reasoning, web_search_call, reasoning, web_search_call, message)
   3 response.web_search_call.searching
   3 response.web_search_call.in_progress
   3 response.web_search_call.completed
   2 response.output_text.annotation.added
   1 response.output_text.done
   1 response.in_progress
   1 response.created
   1 response.content_part.done
   1 response.content_part.added
   1 response.completed                (citations in the final response: 2)
final answer: On **August 4, 2026**, AWS announced the **general availability of Web Search on Amazon Bedrock** for OpenAI's **GPT-5.4, GPT-5.5, and GPT-5.6 Sol, Terra, and Luna** models.

/v1/chat/completions with web_search_options

1. post /v1/chat/completions chat_web_search_options; jq -c '{finish_reason: .choices[0].finish_reason, annotations: [.choices[0].message.annotations[] | .url], content: .choices[0].message.content[0:120]}' chat_web_search_options.out; grep -i '^x-litellm-response-cost:' chat_web_search_options.headers
2. Observed: HTTP 200, two url_citation annotations, grounded answer

{"finish_reason":"stop","annotations":["https://aws.amazon.com/about-aws/whats-new/2026/08/amazon-bedrock-web/","https://aws.amazon.com/about-aws/whats-new/2026/09/amazon-bedrock-web-aws-govcloud/"],"content":"On **August 4, 2026**, AWS announced the **general availability of Web Search on Amazon Bedrock**.\n\nThe new capability:"}
x-litellm-response-cost: 0.097398125

/v1/messages with the web_search tool

1. post /v1/messages messages_web_search; jq -c '{types: [.content[].type], stop_reason, text: .content[0].text[0:120]}' messages_web_search.out; grep -i '^x-litellm-response-cost:' messages_web_search.headers
2. Observed: HTTP 200, grounded text with the AWS source link inline

{"types":["text"],"stop_reason":"end_turn","text":"On **August 4, 2026**, AWS announced the **general availability of Web Search on Amazon Bedrock**.\n\nThe new capability is a built-in"}
x-litellm-response-cost: 0.089045
source link in the text: https://aws.amazon.com/about-aws/whats-new/2026/08/amazon-bedrock-web/

/v1/responses on a Mantle model without Web Search (gpt-oss-120b)

1. post /v1/responses responses_gptoss; jq -c '{types: [.output[].type], tools, answer: ([.output[] | select(.type=="message") | .content[].text] | join(" ") | .[0:120])}' responses_gptoss.out
2. Observed: HTTP 400, Mantle rejects the tool for this model instead of the gateway hiding it

{"error":{"message":"litellm.BadRequestError: BedrockException - {\"error\":{\"code\":\"validation_error\",\"message\":\"Failed to deserialize the JSON body into the target type: ?[0]: Invalid 'tools': unknown variant `web_search`, expected `function` or `mcp` at line 1 column 183\",\"param\":null,\"type\":\"invalid_request_error\"}}"}}

Observations from the run, none caused by this PR:

• Searches are not costed: x-litellm-response-cost-tool-usage stays 0 (left alone, LIT-6870)
• /v1/messages returns citations inline, no web_search_tool_result blocks (left alone)
• Cost map lists Mantle models Mantle 404s: gpt-5.6-cyber, grok-4.6 (pre-existing)
• GovCloud GPT entries carry no web search flag (left alone)

Type

🐛 Bug Fix

Changes

BedrockMantleResponsesAPIConfig filters the Responses tools array against an allowlist of tool types Mantle accepts, and web_search predates AWS shipping it, so map_openai_params dropped it before the request went out. Web Search on Amazon Bedrock went GA on 2026-08-04 as a server-side built-in tool on the OpenAI Responses path for GPT-5.4, GPT-5.5 and GPT-5.6 Sol/Terra/Luna, so the tool now stays in the payload with its external_web_access field untouched, both for a top-level tools entry and for one hoisted out of a Codex additional_tools input item

-_BEDROCK_MANTLE_SUPPORTED_RESPONSE_TOOL_TYPES = frozenset({"function", "mcp", "custom", "namespace", "tool_search"})
+_BEDROCK_MANTLE_SUPPORTED_RESPONSE_TOOL_TYPES: Final = frozenset(
+    {"function", "mcp", "custom", "namespace", "tool_search", "web_search"}
+)

The five GPT entries in the cost map also gain supports_web_search: true so litellm.supports_web_search() and the router's web-search-aware filtering see the capability. Nothing else in the Mantle lineup gets the flag, since AWS scopes Web Search to those models

Four existing tests used web_search as their stand-in for an unsupported tool type, so they now use file_search, which Mantle really does reject and which this config already declares unsupported. The test that pins web_search out of the drop warning reads the warning back through caplog instead of patching the logger

Caveats (if any)

Medium

• Web searches are not costed; AWS bills $12 per 1,000 queries
  • x-litellm-response-cost-tool-usage stays 0 on every leg above
  • The generic built-in tool path prices once per response, not per query, so metering needs its own change (tracked in LIT-6870)

Low

• Mantle models without Web Search now 400 instead of silently dropping the tool
  • Before, gpt-oss-120b answered 200 with a made-up announcement (see the proof above)
  • A model group mixing gpt-oss and GPT-5.x deployments can route a web_search request to the gpt-oss one
  • Workaround: set model_info: {supports_web_search: false} on that deployment, the router honors it
• GovCloud GPT entries stay unflagged for web search
• CircleCI local_testing_part1 and local_testing_part2 are red on tests this PR never touches
  • The two timeout tests fail identically on staging's own pipelines (tracked flake)
  • cohere.command-r-plus-v1:0 tests hit Bedrock's end-of-life error, unrelated to Mantle

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/56ac33bc3eb44a15a96c7022683ff161
Requested by: @mateo-berri

• 8713ba8 passes /live-pr-risk